Artificial neural networks (ANN) are used in computing environments where mathematical algorithms cannot describe a problem to be solved. ANNs are often used for speech recognition, optical character recognition, image processing, and numerous other mathematically ill-posed computation and signal processing problems. ANNs are able to learn by example and, when receiving an unrecognized input signal, can generalize based upon past experiences.
A given ANN is made up, at least in part, of a number of interconnected artificial neuron circuits. The output signal of a given neuron is dependent upon a series of input signals received from a group of other artificial neurons or from sensor or transducer input devices. In the case of a pulse coded ANN, the output signal changes based upon factors like the delay between a pair of input signals. The output of an artificial neuron is often called its “activation.” This “activation” ranges from no or very low activation to high-level activation. In a pulse-coded neuron the level of activation is measured in terms of the frequency and duration of output pulses which are called “action potentials” after the name given to the outputs of biological neurons.
The activation of a pulse-coded artificial neuron is typically based on a nonlinear spatial and temporal sum of its inputs. Spatial sum means the sum of all the inputs that are active at the same time. Temporal sum means that all the inputs are summed with a forgetting factor commonly referred to as a leaky integral. Leaky integrators are key subsystems in pulse-mode artificial neurons as they are responsible for spatially and temporally summing the input signals to determine the artificial neuron's activation.
A given pulse-mode artificial neural network will have dozens to hundreds of leaky integrators. Using conventional approaches, each leaky integrator circuit requires the use of one or more integrated capacitors. Compared to the size of other leaky integrator components, a capacitor is large and requires a substantial amount of space. For example a one picofarad capacitor requires on the order of 2000 square micrometers of space on an integrated circuit fabricated using a standard process. An integrated transistor, in contrast, may require less than ten square micrometers.
If the capacitors can be replaced with smaller components such as transistors, the size of a pulse-mode artificial neuron can be reduced. Moreover, conventional leaky integrators have fixed time constants. If the time constant can be adapted or altered, a pulse-mode artificial neuron could better mimic a biological neuron.